System and method for conducting a search using a wireless mobile device

ABSTRACT

A method and system are provided by which a wireless mobile device takes a vocally entered query and transmits it in a text message format over a wireless network to a search engine; receives search results based on the query from the search engine over the wireless network; and displays the search results.

FIELD OF THE INVENTION

The invention relates to a wireless mobile device based search system,and specifically to submitting queries on a wireless mobile device.

BACKGROUND OF THE INVENTION

The need for faster and easier access to information has never beengreater in this internet era. The popularization of wireless mobiledevices has made it possible to obtain any information wherever andwhenever. Currently, most of the mobile devices on the market havebrowser capability that provides internet access. But navigating throughthe various links and web pages to find the right information is not aneasy task, especially on a small screen of a mobile device. In addition,only a relatively small percentage of the all websites on the internetoffers a Wireless Application Protocol (WAP) version of themselves fordisplay on wireless mobile devices.

Text messaging based information retrieval systems such as the Google™Short Message Service eliminates the need for web pages and browsers ona mobile device by allowing users to send queries and receive answers astext messages. They provide a more dynamic and efficient way to get thedesired information, but a major deficiency of these systems is thatmany of the mobile devices, such as a cellular phone, are tiny and havetiny numeric keypads and screens that make it very difficult to entertext for a Short Message Service (SMS) message. The only text inputmethod available on most involves pressing numeric keys once, twice orthree times to select one of the alphabetical characters associated withit. This makes for very slow input, and discourages users from utilizingfeatures like the Google™ SMS.

The present invention addresses these problems.

SUMMARY OF THE INVENTION

The invention relates to a system and method for obtaining anddisplaying information content from sources external on a wirelessmobile device based on a spoken query.

In one aspect, a method of obtaining query results on a mobile device isprovided. In one embodiment a user is presented with a list of availablequeries on a mobile device such as a cellular phone. The query list canbe customized based on the current location of the user or in responseto a prior search history stored on the mobile device. After the userselects a query verbally, the mobile device prompts the user foradditional information required to complete the query. The user is ableto provide the needed information verbally using speech recognitionsoftware. In one embodiment once the query is complete, it isautomatically converted to a SMS format, addressed to a search engine,and transmitted over the mobile phone network to a search engine. Searchresults based on the query are sent back to the mobile device also inSMS format from the search engine. Once received, the search results arereformatted by the mobile device and displayed on the screen of themobile device.

In another embodiment, the search results are sent to an intermediaryserver on the mobile phone network. In addition to formatting, inanother embodiment the intermediary server also processes advertisementsand appends the advertisements to the search results. In variousembodiments, the added advertisements originate at the search engine; atan advertisement provider on the network; or directly from a particularvendor of goods and services. The formatted search results andadvertisements are then forwarded to the mobile device for display. Inyet another embodiment, upon receiving the search results, the mobiledevice uses text to speech synthesis to generate a voice speaking thereceived search results in addition to displaying them on the screen ofthe mobile device.

In another aspect, the invention relating to a system for obtainingquery results on a mobile device is provided. In one embodiment thesystem includes a wireless mobile device, a wireless network, and asearch engine. According to this embodiment, the wireless mobile deviceincludes a voice input, a speech to text converter, a text to textprotocol converter, a wireless transmitter, and a wireless receiver. Thesearch engine includes a search engine receiver and a search enginetransmitter. The voice input is converted to a message format andtransmitted by the wireless transmitter through the wireless network tothe search engine. The search engine then transmits search results tothe wireless receiver for display.

In another aspect, the present invention relates to a method ofmonetizing a search performed by a wireless mobile device. The methodincludes the steps of speaking a query to the wireless mobile device;transmitting the query from the wireless mobile device over a wirelessnetwork to a search engine; receiving search results by the wirelessmobile device from the search engine over the wireless network;receiving advertisements by the wireless mobile device over the wirelessnetwork; formatting the search results and the advertisements;displaying the search results and the advertisements on the wirelessmobile device; and recovering a monetary remuneration from theoriginator of the advertisements in response to the query. In differentembodiments, the originator may be the search engine, or anadvertisement provider, or a particular vendor of goods and services.

BRIEF DESCRIPTION OF THE DRAWINGS

These embodiments and other aspects of this invention will be readilyapparent from the detailed description below and the appended drawings,which are meant to illustrate and not to limit the invention, and inwhich:

FIG. 1 is a block diagram illustrating a search system according to anembodiment of the present invention;

FIG. 2 is a block diagram depicting the various components of anembodiment of a wireless mobile device as the Input/Output interface ofthe search system;

FIG. 3 is a block diagram depicting the various components of anembodiment of a search engine as a part of the search system;

FIG. 4 is a high-level flow chart illustrating exemplary steps involvedin submitting a query and receiving results by using the search systemaccording to an embodiment of the invention;

FIG. 5 is a flow chart illustrating an example of a dialog between thewireless mobile device and its user for inputting a search queryaccording to an embodiment of the invention; and

FIG. 6 is a block diagram depicting an example of search results inmultiple SMS messages before and after being formatted.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention will be more completely understood through thefollowing detailed description, which should be read in conjunction withthe attached drawings. In this description, like numbers refer tosimilar elements within various embodiments of the present invention.Within this detailed description, the claimed invention will beexplained with respect to preferred embodiments. However, the skilledartisan will readily appreciate that the methods and systems describedherein are merely exemplary and that variations can be made withoutdeparting from the spirit and scope of the invention.

FIG. 1 is an overview of a voice query search system according to anembodiment of the invention. The search system includes a wirelessmobile device 100, a wireless network 102, a search engine 104, andoptionally an intermediary site 106, and a third party advertisementprovider 108. The wireless mobile device 100 is the primary I/O devicethat allows its user to submit a verbal query and displays searchresults based on the query. The wireless network 102 is a channelbetween the wireless mobile device 100 and the search engine 104, wherethe actual search takes place. Search results may be reformatted at thewireless mobile device 100, the search engine 104, the optionalintermediary site 106, or the third party advertisement provider 108.The formatted search results may be combined with advertisementsprovided by the search engine 104, the third party advertisementprovider 108 or preloaded on the wireless mobile device 100 itself.

In various embodiments the wireless mobile device 100 is a cellularphone, a PDA, a BlackBerry®, or any other similar device equipped withvoice recognition technology that allows a user to input informationverbally. The wireless network 102 can be any existing telecommunicationnetwork to which the wireless mobile device 100 is connected. Forexample, if the wireless mobile device 100 is a cellular phone, thewireless network 102 is the mobile phone network providing service tothat particular phone. While connected to the wireless network 102, thewireless mobile device 100 is able to send and receive information toother devices that have access to the same wireless network 102.

After receiving a query over the wireless network 102, the search engine104 generates search results and sends the search results back over thewireless network 102 to the wireless mobile device 100. In addition tothe results of the query, the search engine 104 is also able to sendadvertisements to the wireless mobile device 100. The search engine 104may have the advertisements stored locally or may request theadvertisements for a third party advertisement provider 108.Alternatively the search engine 104 may redirect the third partyadvertisement provider 108 to send the advertisements directly to thewireless mobile device 100.

Before the search results are displayed on the wireless mobile device100, they are formatted to fit on the small screen of such device. Suchformatting may take place on the wireless mobile device 100 itself, atthe intermediary site 106, at the third party advertisement provider108, or the search engine 104. In the process of formatting, additionalinformation such as advertisements or other information can be alsoadded to the search results. The additional information may or may notoriginate at the location where the formatting occurs. The additionalinformation may be obtained concurrently with the search results if theinformation comes from the search engine 104, or may be already storedlocally at the formatting site. The additional information is appendedto the search results and transmitted over the wireless network 102 tothe wireless mobile device 100. In one embodiment, the additionalinformation also serves as the subject of future queries by embedding alink to another function. For example, the link could be a map queryfunction that provides directions to a destination returned as a part ofthe search results. The intermediary site 106 and the third partyadvertisement provider 108 may communicate with each other and with thesearch engine 104 and the wireless mobile device 100 over the wirelessnetwork 102.

Formatting may also occur on the wireless mobile device 100 itself. Forexample, advertisements which are pre-stored on the wireless mobiledevice 100 may also be formatted along with the search results. Inaddition, pre-formatted advertisements provided by external entities maybe sent to the wireless mobile device 100 periodically or in response tothe occurrence of certain events. For example, the issuing of a query orthe movement of the wireless mobile device 100 into a particular areamay trigger a fresh download of advertisements.

After receiving the search results, either directly from the searchengine 104 or from the intermediary site 106, the wireless mobile device100 displays the search results on its screen. In one embodiment, thewireless mobile device uses a text to speech synthesis process togenerate a voice speaking the search results.

The provider of the disclosed technology may collect a license fee forproviding an advertisement platform from the various advertisementsuppliers including the search engine 104, the intermediary site 106,and the third party advertisement provider 108. In addition, revenuegenerated by each of the suppliers from selling advertisements to itscustomers is shared with the provider of the disclosed technology inreturn for utilizing the technology to display advertisements withsearch results on the wireless mobile device 100.

FIG. 2 is a block diagram depicting the various components of a wirelessmobile device 100 according to an embodiment of the present invention.The wireless mobile device 100 includes a voice input 110 such as amicrophone which is electrically connected to a speech to text converter112. The speech to text converter converts 112 the spoken word to a textstring output. The output of the speech to text converter 112 is theinput signal to a text to text protocol converter 114, which processesthe text string into the desired format required for text transmissionover the network. The formatted text is sent to a wireless transmitter116 for transmission over the network 102.

The wireless mobile device also includes a wireless receiver 118 whichreceives the results of the query over the network 102 and whichtransmits the results to a formatter 120 for display on a display 122.In another embodiment the results are transmitted additionally oroptionally to a text to speech synthesizer 124 for audible output 126.

The speech based input mechanism of the wireless mobile device 100generally obviates the need for numerous key presses that wouldotherwise be necessary to address the SMS message, type in its contentsin the proper query format expected by the search engine 104, and toorganize the multiple SMS messages received from a single query. Sincenothing is typed, the user is not aware SMS messaging is being used toissue queries and receive results. The voice recognition and speechsynthesis technology provide a seamless way to input a query without theneed to use the keypad or look at the display of the wireless mobiledevice 100.

In the case where the system does not recognize the response to arequest for more information, the user can type in the requestedinformation. In this embodiment, the formatting of the query utilizesboth the input from the voice recognition portion of the system and theinput from the keypad.

In operation, the wireless mobile device 100 receives via its voiceinput 110 a query spoken by a user. The spoken query is converted by thespeech to text converter 112 to text. The text is then converted by thetext to text protocol converter 114 to a text message. An existingprotocol for transmitting such a text message is SMS, which transmitsshort text messages to and from a mobile device. An alternative protocolis Multimedia Message Service (MMS) which transmits graphics, soundfiles, and video clips in addition to plain text. The text message istransmitted by the wireless transmitter 116 to an external wirelessnetwork 102. When search results become available, they are received ina text message format from the wireless network 102 through the wirelessreceiver 118 of the wireless mobile device 100. The text message is sentto the formatter 120, which formats the search results for display onthe display 122 of the mobile wireless device 100, and the display 122displays them. Formatted information is ordered and presented in aneasily digestible way. The search results in one embodiment aredisplayed in a window that occupies about two thirds of the top portionof the display 122. A scroll bar is shown on the right frame of thewindow if the search results do not all fit in the window.Advertisements and other information are displayed on the remainingportion of the display 122 below the search results.

FIG. 3 is a block diagram depicting the various components of a searchengine 104 as an embodiment of the present invention. The search engine104 includes a search engine receiver 128 and a search enginetransmitter 130, the former receives text messages from the wirelessnetwork 102 and the latter dispatches search results to the wirelessnetwork 102. In communication with the search engine receiver 128 andthe search engine transmitter 130, is the actual search engine whichperforms the search, lists the search results, and generates to therequisite text format.

FIG. 4 is flowchart illustrating exemplary steps in more detail involvedin submitting a query and receiving results according to an embodimentof the invention. FIG. 5 depicts the various displays that would appearon the mobile device screen and/or be audibly spoken by the mobiledevice also according to the embodiment of the invention. Instead ofentering information by using the key pad of the wireless mobile device100, a user speaks a command to such device 100 to initiate a search(Step 134). Upon receiving the command, the wireless mobile device 100displays a list 200 (FIG. 5) of predefined queries which may beselected. The list can be based on the present location of the user orin response to a prior search history stored in the device 100. Anexample of such a list may contain categories such as movie, weather,restaurant and business.

After a selection is made vocally by the user, the wireless mobiledevice 100 continues a dialog with the user to obtain the requisiteinformation to complete the query (Step 136). The wireless mobile device100 may use speech synthesis, pre-recorded speech, and/or visual displayto prompt the user for such information. For example, if the userselects the Restaurant category, the user will then be prompted toprovide the type of restaurant 204 (FIG. 5) and the city or zip code inwhich to search for the restaurant 206 (FIG. 5). If the user fails toanswer properly or if the wireless mobile device is not able torecognize the user's verbal input, the step is repeated until validinformation is provided (Step 138). After a predetermined number offailed attempts to input the query, the wireless mobile device 100returns an error message and the user is given an option to restart fromthe beginning.

If all the required valid information is entered, the user instructs thedevice to send the query 208 (FIG. 5). The flow of the dialog, includingsome of the displayed material, will be predetermined by the wirelessservice provider, the wireless mobile device manufacturer, and/orperiodically updated remotely by them or by the search engine 104. Inthe restaurant search, the information requested and the order in whichthey are requested are provided this way. The actual selections that maybe made in response to each of the requests are dynamically updated. Forexample, the acceptable types of cuisines and geographic limitations ofthe search may be updated when new restaurants are added. After thequery is complete, it is converted first to text (Step 140) and the textto message format (Step 142). The text message is then transmitted overthe wireless network 102 to the search engine 104 (Step 144) and theuser is notified that the query has been sent 210 (FIG. 5).

In one embodiment, the wireless network 102 automatically detects thelocation of the wireless mobile device 100 and passes that informationalong with the text message to the search engine 104. The location maybe used in selecting advertisements to be returned with the searchresults in the following steps. Alternatively mobile devices with GlobalPositioning System (GPS) capability can transfer its location directlyto the search engine without the requirement of mobile networktriangulation.

After the search engine 104 receives the text message specifying thequery, it generates corresponding search results (Step 146), which aresent back to the wireless mobile device 100 (Step 148) over the wirelessnetwork 102 in text message format. The search results are thenformatted (Step 150) and have advertisements added to them on thewireless mobile device 100 (Step 152). In another embodiment, the searchresults are sent to an intermediary site 106 (Step 160), where they areformatted (Step 150′) and appended with advertisements (Step 152′). Theadvertisements originate from either the search engine 104 or anadvertisement provider 106. They are either transmitted via a SMS or MMSmessage over the wireless network 102 or stored in the form of bitmapslocally on the wireless mobile device 100 or the intermediary site 106before being formatted. The selection of advertisements is based onfactors including the particular search results returned and thelocation of the wireless mobile device 100. When a query generatesmultiple search results, each is appended with a different advertisementand sent via separate responsive SMS or MMS messages.

After receiving the search results, either directly from the searchengine 104 (Step 148) or from the intermediary site 106 (Step 162), thewireless mobile device 100 displays the search results on its screen(Step 154). Alternatively, it uses a text to speech synthesis process(Step 156) to generate a voice speaking the search results (Step 158) inaddition to the screen display (Step 154). When there are multiplesearch results received in separate SMS or MMS messages 172, 174 (FIG.6) the messages must be reformatted for a proper display on the screen.Thus the SMS message must be parsed and reformatted for display 176 andthe addition of advertisements 178. The user may use keypad on thewireless mobile device 100 or by giving a voice command, depending onthe output mechanism, to go back and forth between the search results.

The invention may be embodied in other specific forms without departingfrom the spirit or essential characteristics thereof. The presentembodiments are therefore to be considered in all respects asillustrative and not restrictive, the scope of the invention beingindicated by the appended claims rather than by the foregoingdescription, and all changes which come within the meaning and range ofequivalency of the claims are therefore intended to be embraced therein.

1. A method for obtaining query results on a wireless mobile devicecomprising the steps of: speaking a query to a wireless mobile device;converting said query into text; transmitting said text using amessaging protocol over a wireless network to a search engine;performing a search based on said text with said search engine;transmitting search results using said messaging protocol over saidwireless network; formatting said search results; and displaying saidsearch results on said wireless mobile device.
 2. The method of claim 1wherein said messaging protocol is a Short Message Service (SMS)protocol.
 3. The method of claim 1 wherein said messaging protocol is aMultimedia Message Service (MMS) protocol.
 4. The method of claim 1wherein said query and said text are not displayed on said wirelessmobile device.
 5. The method of claim 1 wherein said wireless network isa mobile phone network and said wireless mobile device is a mobilephone.
 6. The method of claim 1 wherein said search results areformatted on said wireless mobile device.
 7. The method of claim 1wherein said search results are transmitted from said search engine toan intermediary site and formatted at said intermediary site.
 8. Themethod of claim 7 further comprising the step of transmitting saidformatted search results to said wireless mobile device from saidintermediary site.
 9. The method of claim 1 wherein said wireless mobiledevice uses a graphic display to display said formatted search results.10. The method of claim 1 further comprising the steps of: receivingadvertisements; formatting said advertisements with said search results;and displaying formatted advertisements with search results on saidwireless mobile device.
 11. The method of claim 1 further comprising thesteps of: synthesizing speech from said formatted search results on saidwireless mobile device; and transmitting said synthesized speech oversaid wireless mobile device.
 12. A method for submitting a query andreceiving results on a wireless mobile device comprising the steps of:speaking to a wireless mobile device a predefined command for starting aquery; speaking requisite information for said query to said wirelessmobile device; converting said information to text; formatting said textfor a messaging protocol to create a message; addressing resultingmessage to a search engine; and transmitting said message over awireless network to said search engine.
 13. The method of claim 12further comprising the step of prompting for said requisite informationby visual prompt.
 14. The method of claim 12 further comprising the stepof prompting for said requisite information by audio prompt.
 15. Themethod of claim 14 wherein said audio prompt is pre-recorded and storedon said mobile device.
 16. The method of claim 14 wherein said audioprompt is synthesized on said mobile device.
 17. The method of claim 12wherein said query is selected from a list of available queries on saidwireless mobile device.
 18. The method of claim 17 wherein said list ofavailable queries is generated in response to a prior search historystored on said wireless mobile device.
 19. The method of claim 17wherein said list of available queries is generated based on thelocation of said wireless mobile device.
 20. The method of claim 17wherein said list of available queries is pre-stored on said wirelessmobile device.
 21. The method of claim 17 wherein said list of availablequeries is downloadable from a remote site.
 22. The method of claim 12further comprising the step of receiving search results over saidwireless network from said search engine.
 23. The method of claim 22further comprising the step of formatting said search results.
 24. Themethod of claim 23 further comprising the step of displaying said searchresults on said wireless mobile device.
 25. A system for making a queryand receiving results over a wireless mobile device comprising: a voiceinput; a speech to text converter in communication with said voiceinput; a text to text protocol converter in communication with thespeech to text converter; a channel; a wireless transmitter incommunication with the text to text protocol converter and said channel;a wireless receiver in communication with said channel; a search enginereceiver in communication with said channel; a search engine incommunication with said search engine receiver; and a search enginetransmitter in communication with said search engine and said channel;wherein said voice input is converted to a message and transmitted bysaid wireless transmitter through said channel to said search engine,and said search engine transmits search results to said wirelessreceiver for display.
 26. A wireless mobile device comprising: a voiceinput; a speech to text converter in communication with said voiceinput; a text to text protocol converter in communication with thespeech to text converter; a wireless transmitter in communication withthe text to text protocol converter and a channel; a receiver incommunication with said channel; and an output device in communicationwith said receiver; wherein said voice input is converted to a messageand transmitted by said wireless transmitter through said channel to asearch engine, and search results from said search engine are receivedby said receiver and displayed by said output device.
 27. A system forreturning query results over a wireless mobile device comprising: meansfor inputting speech; means for converting speech to text; means forconverting said text to a message that is compatible with said textprotocol; means for transmitting said message over a wireless network;means for performing a search based on terms of said message; means forreturning search results over said wireless network; and means fordisplaying said search results.
 28. A method for monetizing a searchperformed on a wireless mobile device comprising: speaking a query tosaid wireless mobile device; transmitting said query from said wirelessmobile device over a wireless network to a search engine; receivingsearch results by said wireless mobile device from said search engineover said wireless network; receiving advertisements by said wirelessmobile device over said wireless network; formatting said search resultsand said advertisements; displaying said search results and saidadvertisements on said wireless mobile device; and recovering a monetaryremuneration in response to said query.
 29. The method of claim 28further comprising the step of receiving said advertisements by saidwireless mobile device over said wireless network.
 30. The method ofclaim 28 whereas said advertisements are pre-stored on said wirelessmobile device.
 31. The method of claim 28 wherein said advertisementsare originated from said search engine and said monetary remuneration isfrom said search engine.
 32. The method of claim 28 wherein saidadvertisements are originated from an advertisement provider andmonetary remuneration is from said advertisement provider.
 33. Themethod of claim 28 wherein said advertisements are originated from aparticular vendor of goods and services and monetary remuneration isfrom said vendor.